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ANNOTATED BIBLIOGEAPHY OF THE 
ADVANCED SYSTEMS DIVISION REPORTS 
(1973 - 1974) 

ABSTRACTS FOR 1969 

1. A'Hearn, F., Daniel, J.N., Enright, C.S., Giggey , F.W., Torr, D.V. , 
& Zappala, A.G, The System Life Cycle; A Defense Weapon System 
Scenario Simulation Model. AFHRL-TR-69-10 , April 1969. AD'-878 
577. 

In order to provide a learning aid with which students in the Defense 
Weapon System Management Center could gain experience in a wide variety 
of decisions in various interrelated functional areas throughout a 
weapon system life cycle, a simulation model was formulated. This 
simulation consists of a life cycle scenario for a fictitious "Con- 
queror" surface to surface missile system, with manual and computer 
assisted student exercises at 35 critical decision points. These 
exercises cover the full range of functional areas and program phasing, 
and are keyed to material and decision techniques covered in concurrent 
classroom instruction. The student receives a "school solution," in 
terms of the next increment of the scenario, at the conclusion of 
each exercise. The scenario and exercise outlines have been completed. 
Recommendations for the preparation of detailed supporting material for 
each exercise and certain unique software are included so. that the 
simulation may be used in the training of weapon system managers. 

2. Askren, W.B., & Newton, R.R, Review and Analysis of Personnel Sub- 
system Test and Evaluation Literature. AFHRL-TR-6 8-7 January 1969. 
AD-859 300. 

This report reviews and analyzes 95 documents related to Personnel 
Subsystem Test and Evaluation. The reports are divided into two groups: 
(1) system tests and (2) related research material. Each report is 
abstracted and then analyzed further in terms of 11 categories: a. 
Scope and Relation to Personnel Subsystem Elements; b. Test Objectives; 
c. Data Requirements and Test Criteria; d. Data Collection Methodology; 
e. Support Requirements; f , Reducing and Analyzing Data; g. Significant 
Test Results; h. Communicating and Using Test Results; i. Factors in 
Planning a Test Program; j. Factors in Conducting a Test Program; k. 
Other Problems , 

Potempa, K,W. A Catalog of Human-Factors Techniques for Testing New 
Systems, AFHRL-TR-68-15 , February 1969. AD-854 482. 

This report contains 16 human-factors testing techniques .contributed 
by a number of Air Force and contractor personnel. While limited in 



The above reports, which were published in 1969, were released to the 
general public in 1973. 
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number, it covers a broad segment of the human-factors testing spectrum. 
For example, the techniques presented range from ttiose used during the 
design and fabrication through those used in field and operational 
system testing. This catalog was developed as a prototype to determine 
its usefulness to human-factors personnel as a reference source for 
human-factors testing devices and techniques. If this prototype finds 
acceptance, a more comprehensive catalog may be developed. The compre- 
hensive catalog would serve two purposes. First, it would present 
information about techniques that have not previously been published. 
Second, it would present in one source most of the techniques currently 
being used in human-factors testing. 



4. Mays, J. A. The Development and Evaluation of a Ultra-High Resolution 
Television System. AFHRL-TR-71-1, February 1971. AD-885 826. 

The report describes the development of a Ultra-High Resolution 
television system, capable of operating at 2047 or 1023 lines/30 frames 
per second, and the evaluation of the television system with three high- 
resolution vidicons. The report describes the electronic circuitry 
developed, in particular the extremely wideband video preamplifier and 
video processor. The evaluation phase of the program includes data 
concerning the performance of the television system and the three 
vidicons. The evaluation found that the television system is capable 
of operating at the proposed 2047 and 1023 TV lines/30 frames per 
second and meets the design goal of 90 MBz video bandwidth. The evalua- 
tion of the three vidicons included: (1) the RCA type 4586, (2) the 
General Electrodynamics type 1347-001 one inch electrostatic deflection, 
electrostatic focus vidicons employing an additional magnetic focus coil 
for improved resolution, and (3) a GE type Z7940 1-1/2 inch electro- 
static deflection, magnetic focus (FPS) vidicon* Each of the vidicons 
was found to have both good and bad qualities, the use of any one of 
them would have to be based upon the requirements of the system. The 
conclusion rea-^.hed by examination of the resultant data supports the 
belief that television systems employing very wide bandwidth and high 
scan rates are quite possible and will result in a substantial advance- 
ment in the state-of-the-art of simulation. 

5. Nagler, A.H. , & Mazurkewitz, A.R. Wide Angle, Infinite Depth-of-Field, 
Optical Pickup for Visual Simulation. AFHRL-TR-71-41, November 1971. 
AD-892 433. 

During Phase I of the program, a study was undertaken to establish the 
most favorable approach toward producing a wide-angle, infinite depth- 
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of-field, inclined-image plane viewing probe for purposes of simula- 
tion. The second phase involved fabrication, test and evaluation of 
an engineering feasibility model of such a device. This report 
describes both phases of the program. Previous optical pickups for 
flight simulators were limited by slant range focus at close approaches 
to a model. Obtaining closer approaches has important size, cost 
and versatility advantages in making simulator terrain models and 
support equipment. Phase I study results indicated the feasibility 
of producing a 140° circular field pickup with full pitch capability 
and a close approach 4.1 mm. A preliminary but realistic design was 
developed and evaluated. The design approach was confirmed by photo- 
graphing a runway model with a modified 110° probe. Other concepts 
examined that had promise of improving resolution were dual sensor 
outputs and dual relays. Component trade-offs, simplifications, and 
techniques were sufficiently developed for the design and fabrication 
phase of the program to proceed directly. The Phase II design, 
fabrication and test program resulted in an engineering model that 
essentially met all of the design goals of the program. The probe 
system was evaluated numerically and photographically, working to an 
altitude of 0.2 inches. It has full functional operation in a static 
form and can readily be reworked to dynamic operation if desired. 



ABSTRACTS FOR 1973 

6. Askren, W.B., Korkan, K.D., & Watts, G.W. Human Resources Sensitivity 
to System Design Tradeoff Alternatives: Feasibility Test With Jet 
Engine Data. AFHRL-TR-73-21, November 1973. AD-776 775. 

The feasibility of developing Design Option Decision Trees to a level 
of detail which shows hardward involved in "on-aircraf t" maintenance, 
and the feasibility of measuring the sensitivity of human resources 
data to design tradeoff problems depicted in these Trees were investi- 
gated. The approach included expanding a portion of an earlier 
developed aircraft jet engine Tree, selecting tradeoff problems from 
the expanded Tree for sensitivity analysis, and collecting psycho- 
metric data from experienced jet engine mechanics regarding this 
sensitivity. Five Design Option Decision Trees were developed for 
turbofan jet engines. Eight turbofan problems were evaluated for 
affect on human resources. The factors of training and experience^ and 
amount of troubleshooting time are affected by choice of design option 
in six of the eight tradeoff problems. Ease of maintenance is affected 
in five problems, and complexity cf tools and equipment is affected in 
two problems. Crew size and job specialty are each affected in one 
problem. 
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7. Askren, W.B. Analysis of Air Force Institute of Technology Course 
#475 '^Laboratory Management of R&D." AFHRL-TR-73-27 , August 1973. 
AD-767 197. 

The Air Force Institute of Technology Course #475, "Laboratory 
Management of Research and Development" was analyzed to determine 
if such a course is needed, what form it should take, and what 
subjects should be included in the curriculum. The analysis also 
provides other supplementary data. It was found that Laboratory 
Coirananders are in favor of retaining the course but believe it 
should be changed. The study recommends that the course be con- 
tinued reoriented to training the new laboratory scientist/engineer 
regarding the duties of his job, that the course be shortened to a 
maximum of 15 days using 21 priority subject matter topics, and 
that a Laboratory Project Engineer Handbook be developed to supple- 
ment the course. The study also recommends that a DOL Course Advisory 
Group be established to monitor the course content and objectives. 
This analysis applies to Course 475 as it was taught through June 1972. 
The course was revised subsequent to that date based partially on the 
findings of this study. Inquiries concerning the current AFIT/SL 
Course 475 content and method of instruction should be addressed to 
the Chief, Continuing Education Division, AFIT School of Systems and 
Logistics, Wright-Patterson Air Force Base, Ohio 45433. 

8. Askren, W.B. Human Resources and Personnel Cost Data in System 
Design Tradeoffs; and How to Increase Design Engineer Use of Human 
Data. Professional paper presented at the 81st Annual Convention 

of the American Psychological Association, Montreal, Quebec, Canada, 
"27-31 August 1973. AFHRL-TR-73-46 , October 1973. AD-770 737. 

A number of studies performed over a period of several years regarding 
the use of human resources and personnel cost data in system design 
tradeoffs were analyzed and the results integrated. Five questions 
were posed and answered. U^at are system design tradeoffs? \^at are 
human resources data? UHiy should military psychologists be interested 
in system design tradeoffs and human resources data? How much effect 
do system design tradeoffs have on human resources and personnel cost? 
And, what does this have to do with increasing engineer use of human 
data in design activities? The following conclusions were derived. 
Tradeoffs are a significant part of the weapon system design process. 
The choice of design alternative in a tradeoff study would, in many 
cases, substantially affect the human resources of the organization 
using the product of the design. It is feasible to use data describing 
these human resources in design tradeoffs. This use could lead to 
development of products which make less demand on those resources. 
Viewing system as a human decision process involving choice points 
and options, gives the psychologist an orientation toward design which 
allows him to more effectively work with the engineer. 



8 

ERLC ^ 



9. Gum, D.R. Modeling of the Human Force and Motion-Sensing Mechanisms 



AFHRL-TR-72-54, June 1973. AD-766 444. 



The purpose of the study was: to investigate human force and motion- 
sensing mechanisms; to develop models for, the prominent or potanti- 
ally artificially stimulatable mechanisms; to implement them on an 
analog computer; and to investigate their responses to various 
force and motion-forcing functions. Models were implemented and 
tested for a semicircular canal, the otolith, head motion muscle 
spindle sensing, and body seat pressure sensing. The relative 
magnitude of the sensed force and motion through the various 
mechanisms has not been possible to assess because the action of some 
mechanism transducers, i.e., the Pacinian receptors, and the proces- 
sing of the information received from the various receptors is not 
well understood. However, tests of the models have demonstrated the 
relative time delays between applied force and perceived force for the 
various mechanisms, showing that both the muscle spindle and pressure- 
sensing mechanisms perceive an applied force much more rapidly than the 
vestibular system. Also, the long adaptation phenomenon associated 
with the semicircular canals which seems to degrade their usefulness 
in flight and the rapid adaptation phenomenon associated with the 
pressure sensors which makes them important sensors for consideration 
in the design of motion systems have been shown through model testing. 

10. Joyce, R.P., Chenzoff, A.P. , Mulligan, J.F., & Mallory, W.J. Fully 
Proceduralized Job Performance Aids; Draft Military Specification 
for Organizational and Intermediate Maintenance. AFHRL-TR-73-43(I) , 
December 1973. AD-775 702. 

This volume supplies a model for specifications for the preparation of 
Fully Proceduralized Job Performance Aids for the maintenance of Air 
Force equipment. The model reflects the research findings of the Air 
Force Human Resources Laboratory and other Department of Defense 
agencies concerning maintenance data. It has the unique feature of 
requiring that a task analysis yielding certain intermediate products 
necessary, for the development of this type of data be prepared in a 
standard format and submitted for review by the Procuring Agency. 
These intermediate products include items such as an Annotated Task 
Identification Matr.ix, a Test Equipment and Tool Use Form, and a 
Generalized Task List, as well as step description worksheets and level 
of detail requirements. The aids to be developed from these specifica- 
tions are for the organizational and intermediate maintenance functions: 
adjust, align, calibrate, checkout/troubleshooting, clean, dissassemble/ 
assemble, inspect, lubricate, operate, remove/ ins tall , repair, and 
service. The specification, covers all content, writing, illustration, 
and format requirements for Job Guide Manuals and Fully Proceduralized 
Troubleshooting Aids. It includes comprehensive quality assurance 
provisions and an approved verb list. 



ERIC 



12 



11. Joyce, R.P., Chenzoff, A. P., Mulligan, J.F., & Mallory, W.J. Fully 
Proceduralized Job Performance Aids: Handbook for JPA Developers. 



AFHRL-TR-73-43(II), December 1973. AD-775 705. 



This volume provides guidance for the development of Fully Proce- 
duralized Job Performance Aids (JPAs) for the organizational and 
intermediate maintenance of Air Force equipment. It contains detailed 
instructions to the contractor for preparing fully proceduralized JPAs 
in accordance with the Draft Specification (Volume I of this technical 
report). It includes instructions for performing the behavioral task 
analysis, for development of the Job Guides and Fully Proceduralized 
Troubleshooting Aids (FPTAs) , and for insuring quality and accuracy 
assurance. The very important validation and verfication activities 
are also thoroughly discussed. Personnel types and qualifications 
required for efficient development of JPAs are also included. In 
addition, this volume presents a strategy and guidance for developing 
supervised practice exercise designed to produce the skills required 
for preparing fully proceduralized JPAs. 

12. Joyce, R.P., Chenzoff, A. P., Mulligan, J.F. , & Mallory, W.J. Fully 
Proceduralized Job Performance Aids: Handbook for JPA Managers and 
Training Specialists. AFHRL-TR-73-43(lII) , December 1973. AD-775 
706. 

This report provides guidance for the Air Force Data Managers charged 
with the responsibility for the procurement of Fully Proceduralized 
Job Performance Aids (JPAs). It provides guidelines, suggested proce- 
dures, and checklists for use by data managers in the review and assess- 
ment of the subproducts, intermediate proc^ucts, and JPAs produced in 
accordance with the draft specification contained in Volume I of the 
Technical Report. It also provides guidance to Air Force Training 
Specialists in development of training coordinated with JPAs, and in 
specifying the appropriate JPA/training trade-off for new equipment 
systems. 

13. Knoop, P. A., & Welde, W.L. Automated Pilot Performance Assessment 
in the T-37: A Feasibility Study. AFHRL-TR-72-6 , April 1973. 
AD-766 446. 

Research was conducted to develop a capability for quantification and 
assessment of in-flight pilot performance for utilization in Under- 
graduate Pilot Training (UPT) . This feasibility effort was directed 
to overcoming the disadvantages of the traditional subjective rating 
of a pilot trainee's performance by the instructor pilot. This was 
accomplished through the development of an automated, objective per- 
formance measurement system that possesses the characteristics of 
reliability, validity, and sensitivity. A T-37B was instrumented to 
digitally record 24 flight and engine parameters. An extensive computer 
software system was developed with which to reduce, calibrate, and 
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analyze the recorded data from the lazy 8 and barrel roll maneuvers, 
and compute performance measures . Criterion values for the two 
maneuvers were developed by utilizing task analysis data, narrative 
descriptions, and recorded in-flight maneuver performance of a 
highly qualified Air Training Command instructor pilot. Utilizing 
recorded data from 16 students and 4 instructors, experimental 
performance measures were derived through an interative analytic 
approach. Study results indicated that lazy 8 performance assess- 
ment can be accomplished using the flight parameters of roll angle, 
pitch angle, and airspeed in a single summary error measure. Barrel 
roll measuremnt is dependent upon roll and pitch angle, acceleration 
(g force), and roll rate. A definite relationship between, roll and 
pitch was determined to be critical to measurement. Discussions of 
measurement validation methods, debriefing plots, a sampling rate 
study, instrumentation techniques, and problem areas are provided. 

Knoop, P. A. Advanced Instructional Provisions and Automated Per- 
formance Measurement . Human Factors , 15(6), 583-597, December 1973. 

An advanced siir.ulation research system is being developed to support 
experiaments oriented toward quantifying the effect on transfer of 
training of alternative training and simulation techniques. Included 
in the system is sophisticated training research software which 
automates most of the functions traditionally performed by flight 
simulator instructors or operators. Also under development are 
techniques for automatically assessing pilot proficiency in the 
simulator and in the operational aircraft. Recent feasibility studies 
in performance measurement resulted in identification of necessary 
flight variables for assessing proficiency on two undergraduate pilot 
training maneuvers; established alternative approaches to developing 
measurement techniques on a broader scale; applied existing technology 
to develop an in-flight data acquisition system for the T-37 aircraft; 
and raised (or, more correctly, reraised) issues regarding standard- 
ization in instruction and rating procedures. 

Lintz, L.M., Loy, S.L., Brock, G.R. , & Potempa, K.W. Predicting 
Maintenance Task Difficulty and Personnel Skill Requirements Based on 
Design Parameters of Avionics Subsystems. AFHRL-TR- 72-75 , August 
1973. AD-768 415. 

The relationships among subsvstem design characteristics, personnel 
skill characteristics, personnel skill characteristics, and job 
performance were investigated for avionics subsystems. A list of desi 
characteristics was estiablished, based on expert opinions of avionics 
engineers. Air Force supervisors, and instructors. Functional loops 
and line replaceable units were selected from ten subsystems represent 
ing navigation, flight control, communications, and fire control 
subsystems. Experienced maintenance supervisors identified high skill 
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and low skill maintenance personnel. The same supervisors associated 
performance times and error probabilities with these personnel for 
three maintenance tasks - an easy task, a difficult task, and a 
complete functional checkout task. Supervisors also rated each task 
on a scale difficulty. Both stepwise regressions and factor analysis 
were used to derive equations for predicting performance time, error 
probability^ and task difficulty from design characteristics. Similar 
analyses derived equations relating performance time and errors to 
personnel characteristics. Multiple correlation coefficients were 
from 0-50 to 0.93. Personnel factors of Experience, Aptitude, Breadth 
of Skills, Motivation, Training, Time in Grade, and Non-AF Technical 
Education were identified. Equipment design factors of Checkout 
Complexity, Checkout Information, Length of Checkout, Accessibility, 
Equipment Complexity, and Test Equipment and Adjustments are associated 
with performance times and errors. 

16. Mays, J. A., & Irish, K.M. The Development and Evaluation of an Optimized 
Video Output From a Wide Angle Optical Probe. AFHRL-TR-73-22 , 
December 1973. AD-774 577. 

This report describes a two-phase program to develop an optimized video 
output from a wide angle optical probe. Phase I of the program presents 
data resulting from radiometric and photometric studies of model 
paints, textures and illumination sources; performance data resulting 
from tests conducted on the wide angle optical probe; and a preliminary 
system design. Phase II of the program dealt with system development 
and evaluation. The report describes the detailed design of the high 
resolution 2-inch vidicon camera, including particular emphasis on 
the wideband video preamplifier and video processing electronics. The 
evaluation phase of the program includes data concerning the performance 
of the television system, and the performance of the integrated system 
including the optical probe, magnifying image intensifier and ultra 
high resolution 2-incli vidicon camera. The conclusion reached by 
examination of the resultant data supports the belief that a truly 
optimized video output from a wide angle optical probe has been achieved. 

17. Mitchell, R.L., Lucero, A.B., & Harrison, R.E. Simulation of High- 
Resolution Radar Displays of City Complexes. AFHRL-TR-73-11, 
August 1973. AD-768 413. 

The modeling and simulation techniques capable of generating large- 
scale high-resolution radar displays of cultural features are investi- 
gated. Specific features include power transmission towers, buildings, 
railroads, bridges, storage tanks, streets, rivers, and natural terrain. 
In addition, the general background-type of return is also discussed. 
The problem of real-time simulation is assessed. The techniques have 
been applied to simulate radar images of a 2,25 x 2.25 mile area of 
North Long Beach, California, to a resolution of 20 feet.- 
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18. Schumacher, S.P. Development of a Technical Data File on the Design 
and Use of Instructional Systems. AFHRL-TR-73-41, December 1973. 
AD-775 149. 



This report describes the development of a technical data file con- 
'-erned with the technology of Instructional System Development suitable 

or a variety of users. The file was prepared in a way amenable to 
later computerized storage and retrieval. General information sources 
and indexes of highly probable relevant content were reviewed with key 
words and relevant specialty journals covering the period 1950 to 1973 
were searched. Abstracts of articles providing opinion, new methodology, 
evaluative summary and literature review were of a summarizing 
descriptive nature. Abstracts of articles reporting sampling studies, 
correlational analysis, and experimental data were prepared more 
comprehensively so that they might often be used in lieu of the article. 
A common format was used with a bibliographic/indexing information page 
and an evaluation checklist being included. The principal results of 
the effort are as follows: (1) a paper file of 2,693 abstracts, (2) 
a card file of titles, (3) a coordinate index, (4) a comprehensive key 
word index and bibliography, and (5) MT/ST tapes of 1,950 of the 
abstracts. Incidental to the development, comprehensive guidelines 
for abstracting this type of literature and a compendium of author- 
noted research-and-development needs were prepared. 

19. Valverde, H.H. A Review of Flight Simulator Transfer of Training 
Studies. Human Factors , 15(6), 510-523, December 1973. 

Often operational equipment is considered to be the most effective and 
valid training equipment. However, this is not true in every instance. 
In fact, sometimes it may be undesirable to use real equipment for 
training if suitable simulators are available. For example, the use 
of operational equipment has several disadvantages which include (1) 
high costs, (2) limitation on practice of varied aspects of tasks, and 
(3) safety hazards. Practical decisions in the use of training devices 
depend upon compromises between economic and training objectives. A 
trainer need not duplicate operational equipment to have training value. 
Technical reports pertaining to flight simulator transfer of training 
studies available from the Defense Documentation Center are reviewed. 

ABSTRACTS FOR 1974 

20. Askren, W.B., and Korkan, K.D. Design Option Decision Tree: A Method 
for Schematic Analysis of a Design Problem and Integration of Human 
Factors Data. In Proceedings of Human Factors Society 18th Annual 
Meeting, Huntsville, Alabama, October 1974. 

A Design Option Decision Tree (DODT) is a graphic means of showing 
the design options available at each decision point in the design pro- 
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cess. Several examples of DODTs for aircraft design problems are 
shown. The procedures for developing a DODT are described. A pro- 
posed method for use of the DODT to resolve a design problem is 
presented. This method includes evaluating the design options in 
the Tree for impact on the system, and tracing paths through the Tree 
as dictated by specific design goals. The use of human factors data 
as one of the evaluation parameters is illustrated. The paper 
concludes with a discussion of other uses of a DODT. 

21. Askren, W.B. Preview of the Special Issue of Human Factors on "Human 
Resources as Criteria for System Design and Organization Planning." 

In Proceedings of Human Factors Society 18th Annual Meeting, Huntsville, 
Alabama , Oc tob er 1974 . 

A preview of a special issue of the Human Factors journal is presented. 
The issue will be published in early 1975 on the topic, "Human Resources 
as Criteria for System Design and Organization Planning." The theme of 
the special issue, and the content of each of the articles is described. 

22. Baran, H.A. Air Force Human Resources Laboratory Military Personnel 
Costing Conference . Proceedings of the AFHRL Military Personnel 
Costing Conference hald 2-3 May 1973 at the Advanced Systems Division, 
Wright-Patterson AFB, Ohio. AFHRL-TR-74-106 , December 1974. 

This conference was the initial step in an R&D project to develop and 
demonstrate a family of techniques to generate standard personnel cost 
data for use in: (a) weapon system design engineering; (b) weapon system 
life cycle support operations; and (c) personnel/manpower budgeting. 
It was organized to provide technical guidance in directing the efforts 
of this project. Representatives from various organizations within Air 
Force Headquarters, Tactical Air Command, Air Force Systems Command, Air 
Training Command, Air Force Logistics Command, and the US Naval Person- 
nel Research Laboratory comprised the membership. The objectives were: 
(A) identify and summarize the various concepts and practices of person- 
nel costing and how they relate to system engineering design, system 
support, and various command level personnel, manpower, and budgetary 
activities; (B) identify the "users" of personnel cost data, their 
present requirements and uses for it, and their desires for new types 
or formats of such data; (C) ascertain what has been accomplished by 
other agencies that could be adapted to satisfy in part/whole the 
requirements identified in objectives B and identify work currently in 
progress which might be applicable; and (D) summarize the most pressing 
research goals and requirements. 

23. Connelly, E.M. , Loental, D.G., Bourne, F.J., & Knoop, P. A. Computer- 
Aided Generation of Performance Measures for Man-Machine Systems. In 
Proceedings of Human Factors Society 18th Annual Meeting, Huntsville, 
Alabama, October 1974. 

The design, application, and evaluation of man-machine systems are 
limited by our ability to measure system performance in a reliable and 
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sensitive manner. Without adequate performance measures, there is no 
way to test system designs, to plan and execute training systems, or to 
effectively evaluate operational systems. Typically, measures are 
manually produced by selecting a set of candidate performance measures 
which are subsequently tested for reliability and validity. Since the 
measurement value of a given candidate measure is not known until these 
tests are complete, this process, which may be an iterative process, 
can be both time consuming and costly. Also, since only a few candidate 
measures can be investigated manually, there is a high probability that 
superior measures are not even considered. Automating at least some of 
the manual operations required can result in^ improved performance 
measures in less time and at lower cost. The performance measurement 
generating processor described in this paper accepts demonstration data 
representing various levels of performance, and under user control, 
analyzes the data to provide candidate performance measures. The pro- 
cessor also conducts validation tests and orders candidate measures 
according to their measurement value. Output from the FORTRAN IV 
processor includes results from validation tests and specifications 
for objective performance measures. 

Connelly, E.M., Bourne, F. J. , Loental, D.G., & Knoop, P. A. Computer- 
Aided Techniques for Providing Operator Performance Measures. AFHRL- 
TR-74-87, December 1974. 

This report documents the theory, structure, and implementation of a 
performance measurement processor (written in FORTRAN IV) that can 
accept performance demonstration data representing various levels of 
operator's skill and, under user control, analyze data to provide 
candidate performance measures and validation test results. The proces- 
sor accepts two types of information: (1) sample performance data on 
magnetic tape and (2) user information reflecting knowledge about 
features of the performance that are considered to be important to 
measurement. The sample performance data input is smoothed by the 
processor in order to remove or reduce noise factors in accordance with 
information provided by the user. Criterion performance functions are, 
optionally, provided by the user or are computed by the processor using 
skilled performers' data. The processor then develops a discrete repre- 
sentation of continuous performance data based on observed deviations 
from the criterion functions. This discrete representation, in turn, is 
used to model each performance using state-space techniques. The 
processor operates on the state-space model to compute vectors which 
form generators of various conceivable measure spaces. Candidate per- 
formance measures are then generated by operating on the vectors with 
multiple regression algorithms. Empirical validation tests of several 
types are applied to the candidate measures for assessment of their 
validity-likelihood. The processor can be applied to measurement, 
problems where the human operator working with his equipment obtains 
demonstrations of various levels of performance. These potential 
applications include those situations where criterion performance cannot 
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be quantitatively predefined and/or the existing definitions are 
ambiguous. Demonstration of some portions of the processor was 
accomplished using limited flight demonstration data from an 
instrumented T-37B aircraft for five undergraduate pilot training 
(UPT) maneuvers: (1) Barrel Roll, (2) Lazy 8, (3) Cloverleaf, (4) 
Split S, and (5) Normal Landing. 

25. Connelly, E.M. , Bourne, F.J., Loental, D.G. , Migliaccio, J.S., 
Burchick, D.S., & Knoop, P. A. Candidate T-37 Pilot Performance 
Measures for Five Contact Maneuvers . AFHRL-TR-74-88, December 
1974. 

The objective of this program was to develop candidate pilot per- 
formance measures for five undergraduate pilot training (UPT) contact 
training maneuvers flown in the T-37B aircraft. The work included 
development and application of a method of analyzing operator per- 
formance tasks for purposes of identifying candidate measures. This 
resulted in sectoring of each T-37B maneuver into functional segments, 
wherein the dominant measurement variables are consistent, and task 
segments, wherein the relationships among the dominant measurement 
variables are consistent. Several types of measures were then defined 
which, collectively, satisfy measurement needs over all task segments. 
Specific candidate measurement formulae were developed for each seg- 
ment in accordance with the analysis results. Computer programs 
(FORTRAM IV) were developed and implemented to: (1) smooth, print out, 
and plot data recorded on-board a T-37B aircraft; (2) automatically 
detect task segment boundaries; (3) compute criterion function from 
skilled performer's data; (4) compute measures specified at run-time 
by the user; and (5) perform and print results of several empirical 
validation tests of the candidate measures for subsequent researcher 
analysis . 

26. Foley, J. P. Evaluating Maintenance Performance: An Analysis. AFHRL- 
TR-74-57(I), October 1974. AD-A004 761. 

Late in 1962, the writer prepared a paper entitled Performance Testing: 
Testing for What is Real (Foley, 1963). This was followed by a Biblio- 
graphy on Maintenance Personnel Performance Measurement by Askren (1963) 
and work on a draft of an annotated bibliography by Porterfield in the 
same year. All of these efforts indicated that there was probably a 
lack of job realism in the formal measuring devices used to ascertain 
the training success and promotion potential of maintenance personnel. 
They also indicated that although a rather extensive technology had 
been developed for paper and pencil testing, no well-structured technol- 
ogy or guidance existed for the development and administration of job 
performance tests. These preliminary works and findings indicated a 
requirement for exploratory development concerning job performance tests 
-for maintenance. To avoid duplication in such an exploratory development 
program, the approach was to tap the wealth of existing, but scattered, 
sources of available hard data concerning job performance measurements, 
to structure these data as they applied to the measurement of ability to 
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perforin electronic maintenance tasks, to analyze them in relation to 
current Air Force practice and to make recommendations for the develop- 
ment and tryout of effective job performance measurements for Air Force 
electronic maintenance. Paper and pencil testing procedures are used 
almost exclusively for determining which personnel arc sleeted for 
training, for determining student progress while in training and for 
determining the promotion eligibility of personnel assigned to field 
maintenance units. A number of studies are cited which indicate that 
low correlations were obtained by comparing job task performance tests 
to paper and pencil theory tests and to job knowledge tests. Several 
studies also are cited which indicate that the traditional theory 
content, found in most electronic maintenance training progratns, does 
not contribute a great deal to the ability to perform electronic main- 
tenance tasks. A full application of the modern technology for tech- 
nical training development would solve the course content problem. This 
technology requires a systems approach to training program development 
in which training objectives are based on a complete job task identifi- 
cation and analysis. Criterion referenced Job Task Performance Tests 
(JTPT) are required to determine if training objectives are achieved. 
However, a serious gap remains in this technology since adequate guidance 
is not available for the development of JTPT. As a result of this 
analysis, it was recommended that comprehensive exploratory and advanced 
development efforts concerning Air Force maintenance should be established 
and funded. These programs should systematically and comprehensively 
identify and solve problems concerning maintenance practice in the field 
and problems concerning the selection and training of maintenance 
personnel. A necessary first effort should be to gather and study hard 
data on how well maintenance men can perform the key tasks of their jobs. 
Also 5 based on the results of this analysis a contractual effort was 
initiated to develop criterion referenced JTPT for electronic maintenance 
tasks. This effort was followed by an attempt to develop both graphic 
and video symbolic substitutes of improved empirical validity. The 
results of these efforts are reported in Volumes II, III, and IV of this 
series of documents. 

27. Foley, W.L. High-Speed Rotating Mirror System for Wide-Angle Image 
Projection. AFHRL-TR-74-42 , June 1974. AD-AOOl 629. 

The design and fabrication of visual display systems with a wide field- 
of-view and high resolution, as is required for pilot training simulators, 
is a formidable problem. Individual displays must be carefully 
fabricated and mosaicked together so the resulting system maintains 
continuity and resolution over the total display field. While such 
systems have been successfully developed, the optical components can be 
large and expensive, and alignment is a formidable problem where several 
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channels are required. Methods were investigated for wide angle 
deflection of a laser beam at rates compatable with standard video 
processing. The effort consisted of investigation and design of a 
magnetic suspension and rotation device for achieving scan rates 
in excess of 3,000 revolutions per second. Two design ("Q" coil 
and differential pickup) approaches were investigated and measure- 
ments were made of rotational rates and acceleration verses input 
drive power. Stability was investigated under short term conditions 
for different rotor configurations, and special measures were under- 
taken to minimize drift of the supported body. One of the designs, 
namely the differential pickup type, appears worthy of further 
investigation and testing. 

28. Hicks, V.B., & Tetmeyer, D.C. Simulating Maintenance Manning for 
New Weapon Systems: Data Base Management Programs . AFllRL-TR-74- 
97(IV), December 1974. AD-AOll 989. 

The Air Force Human Resources Laboratory, in conjunction with Head- 
quarters Air Force System Command and the Aeronautical Systems Di'^^ision, 
has developed and implemented a new method for determining and 'evaluat- 
ing the maintenance manning required for new weapon systems. Key 
features of the new approach are: 1. Manning requirements are deter- 
mined by dynamic simulation of maintenance work to support specific 
operational scenarios. The number of people needed to support the 
same aircraft and flying hour program varies greatly with basing and 
deployment, mission types, and sortie length. 2. Manning is based on 
required availability, not just on the sum of manhours worked. While 
worktime is a major consideration, equipment does not break on schedule 
and combat aircraft cannot always fly on a schedule that provides an 
even workload. 3. Maintenance frequencies and job requirements for a 
new system are extrapolated from Air Force experience with similar 
equipment, adjusted for differences in design and environment. The 
total time a man is tied up on a job is explicitly modeled, not just 
'Vrench turning*' time. 4. Output is in a format compatible with the 
Air Force Manpower Authorization System. This approach provides more 
realistic and useable estimates of manning requirements and focuses on 
how both a contractor's design and Air Force procedures contribute to 
maintenance cost. As soon as the new aircraft configuration can be 
defined, comparable equipment and design differences are identified 
at subsystem level. Maintenance data on comparable equipment is 
processed through a series of specially designed programs. These 
data are the basis for developing initial estimates of frequency, time, 
and resources for each maintenance task when modified by the judgement 
of experienced maintenance technicians and project engineers. The 
logistics and operational concepts basing, maintenance organization 
and flying schedules are developed in detail for various scenarios 
and set down in explicit form for input to the simulation. The 
simulation attempts to generate the specified missions in the computer. 
Aircraft are turned and maintenance accomplished as they break or need 
service according to the task data specified in the input. Manning 
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levels in the various work centers are constrained on successive runs 
until further reduction prevents the necessary number of missions 
from getting off. Computer programs used in the final stage of the 
maintenance manpower simulation modeling process integrate direct work 
center manning output from a number of simulation runs; add supervision, 
overhead requirements, and standard element manning; and apply appro- 
priate grade spread criteria. The output is a complete basic author- 
ization document for maintenance manning, compatible with Air Force 
manpower authorization procedures. The program may be exercised to 
obtain manning for any proposed flying rate with the siirulated range 
and has an option for adjusting existing equations with new data 
representing updated or alternative conditions. The series of COBOL 
and FORTRAN computer programs described in this volume was developed 
to permit a Logistics Composite Model (LCOM) maintenance data base to 
be constructed in simplified format with all the data describing a 
single task contained on one punch card or magnetic file record. The 
programs translate the task data into the necessary LCOM inputs and 
incorporate a number of useful diagnostic checks which eliminate the 
more frequently encountered data preparation errors. Use of these 
programs implifies data base development, update, and control. They 
are c:arrently in operation on the Aeronautical Systems Division's 
CDC 6600 computer. The methodology and models have been successfully 
applied on the A~10 program and are being considered for implementation 
on other aeronautical systems in or entering development. Work is 
underway in conjunction with Air Force Logistics Command to incorporate 
this methodology into a system for total logistics trade-offs and life 
cycle costing. 

29. Irish, K.M., & Orszulak, J.H. Visual Simulation Video Processing 
Techniques . AFHRL-TR-74-76 , December 1974. AD-A009 640. 

This report describes a two-phase program to further the development of 
visual simulation video processing techniques as applied to a wide angle 
optical probe/TV camera. Phase I of the program included the study of 
applicable wide-angle display techniques, study of additional processing 
of the probe's video signal for proper mating to a display, study of 
color visual requirements, and a review of state-of-the-art separation 
techniques for TV cameras. Phase II of the program dealt with the 
construction of' a video matrix generator to provide the necessary 
electronic processing to mate the optical probe/TV camera system to 
applicable wide angle displays. The conclusions reached were that 
a matrix of display elements is necessary for a compact and realistic 
wide-angle display, and a cole TV camera could be developed using 
present technology to mate with the Farrand wide-angle optical probe. 
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30. Joyce, R.P., & Chenzoff, A. P. Improving Job Performance Aids Through 
Condensation, Dual-Level Presentation, Promotion of Learning and Entry 
by Malfunction Symptoms. AHlRL-TR-74-12 , March 1972. AD-781 757. 

This report describes ^the research effort that attempted to simplify 
and condense the presentation of Job Performance Aid (JPA) data. It 
also describes a method of presenting technical data to both experienced 
and inexperienced personnel in one JPA format. Such factors as text 
limitations, illustration criteria, layout restrictions, and physical 
size of the manuals were found to contribute to the excessive bulk of 
the JPAs. Controls were developed for each of the influencing factors. 
Dual-level presentation of technical data was also studied. This 
portion of the study identified the types of information required, and 
not required by experienced technicians. The original JPA sample was 
reformatted, incorporating both condensation principles and dual-level 
presentation factors. The application of condensation principles 
resulted in a decrease from 104 pages in the original 4*' x 8" size to 
18 pages in the larger 8 1/4" x 10 3/4" size, and information was 
provided for both experienced and inexperienced personnel. The sample 
JPAs were then evaluated by novice and experienced mechanics. The 
condensed, dual-level JPAs provided proper information to both experi- 
enced and inexperienced personnel and were well received. A third 
aspect of this study concerned troubleshooting. A review of the goals 
of fully proceduralized troubleshooting was accomplished, and possible 
changes in goals were suggested. A possible way to enter a trouble- 
shooting procedure utilizing known failure symptoms was developed and 
discussed. 

31. Maher, F.A. , & York, M.L. S imulating Maintenance Manning for New 
Weapon Systems: Maintenance Manpower ^ianagement During Weapon 
System Development . AFHRL-TR-T4-97 (I) , December 1974. AD-AOll 986. 

The Air Force Human Px.esources Laboratory, in conjunction with Head- 
quarters Air Force Systems Command and the Aeronautical Systems Division, 
has developed and implemented a new method for determining and evaluating 
the maintenance manning required for new weapon systems. Key features of 
the new approach are: 1. Manning requirements are determined by dynamic 
simulation of maintenance work to support specific operational scenarios. 
The number of people needed to support the same aircraft and flying hour 
program varies greatly with basing and deployment, mission types, and 
sortie length. 2. Manning is based on required availability, not just 
on the sum of ma^-hours worked. While worktime is a major consideration, 
equipment does not break on a schedule and combat aircraft cannot always 
fly on a schedule that provides an even workload* 3. Maintenance fre- 
quencies and job requirements for a new system are extrapolated from 
Air Force experience with similar equipment, adjusted for differences 
in design and environment. The total time a man is tied up on a job is 
explicitly modeled, not just "wrench turning" time. 4. Output is in a 
format compatible with the Air Force manpower authorization system. 
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This approach provides more realistic and useable estimates of manning 
requirements and focuses on how both a contractor's design and Air 
Force operational procedures contribute to maintenance cost. As soon 
as the new aircraft configuration can be defined, comparable equipment 
and design differences are identified at subsystem level. Maintenance 
data on comparable equipment is processed through a series of specially 
designed programs. These data are the basis for developing initial 
estimates of the frequency, time, and resources for each maintenance 
task when modified by the judgement of experienced maintenance tech- 
nicians and project engineers. The logistics and operational concepts, 
basing maintenance organization, and flying schedules are developed in 
detail for various scenarios and set down in explicit form for input 
to the simulation. The simulation attempts to generate the specified 
missions in the computer. Aircraft are turned and maintenance accom- 
plished as they break or need service according to the task data 
specified in the input. Manning levels in the various work centers are 
constrained on successive runs until further reduction prevents the 
necessary number of missions from getting off. The results of various 
simulation runs are integrated in a final series of computer programs 
that computes manning in each work center for any flying program within 
the simulated range, adds authorized overhead, adjusts for grade level 
limitations, and produces a complete basic authorization document for 
maintenance manning in the required format. The methodology and models 
have been successfully applied on the A-10 program and are being 
considered for implementation on other aeronautical systems in or 
entering development. Work is underway in conjunction with Air Force 
Logistics Command to incorporate this methodology into a system for 
total logistics trade offs and life cycle costing. 

Miller, R.B. A Method for Determining Task Strategies. AFHRL-TR-74-26, 
May 1974. AD-783 847. * 

This study is aa extension of task analysis methodology. The concept of 
strategic principles or task strategies implicit in the job activities 
of highly proficient performers is examined with the Intent of improving 
training. The goal is to identify or invent such strategies, as appro- 
priate, and then proceed with training encouraging use of them. A set 
of 25 information processing functions is described along witff examples 
of strategic principles and training implications. Certain other 
strategic principles not readily interpretable in information processing 
terms are presented also. An analytic procedure for determining and/or 
devising strategies is provided and suggestions on the teaching and 
learning of strategies are summarized. 

Moody, W.D., Nichols, S.R., & Tetmeyer, D.C. Simulating the Maintenance 
Manning Required for New Weapon Systems; Vol V - Manpower P r ograms. 
AFHRL-TR-74-97(V), December 1974. /J)-A011 990. ^ 

The Air Force Human Resources Laboratory, in conjunction with Head- 
quarters Air Force Systems Command and the Aeronautical Systems Division, 
has developed and implemented a new method for determing and evaluating 
the maintenance manning required for new weapon systems. Key features 
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of the new approach are: 1. Manning requirements are determined by 
dynamic simulation of maintenance work to support specific opera- 
tional scenarios. The number of people needed to support the same 
aircraft and flying hour pro^^ram varies greatly with basing and deploy- 
ment, mission types, and sortie length. 2. Manning is based on 
required availability, not just on the sum of manhours worked. While 
worktime is a major consideration, equipment does not break on 
schedule and combat aircraft cannot always fly on a schedule that 
provides an even workload. 3. Maintenance frequencies and job re- 
quirements for a new system are extrapolated from Air Force experience 
with similar equipment, adjusted for differences in design and environ- 
ment. The total time a man is tied up on a job is explicity modeled, 
not just ^'wrench turning" time. 4. Output is in a format compatible 
with the Air Force Manpower Authorization System. This approach 
provides more realistic and usable estimates of manning requirements 
and focuses on how both a contractor's design and Air Force procedures 
contribute to maintenance cost. As soon as the new aircraft configura- 
tion can be defined, comparable equipment and design differences are 
identified at subsystem level. Maintenance data on comparable equip- 
ment is processed through a series of specially designed programs. 
These data are the basis for developing initial estimates of the fre- 
quency, time, and resources for each maintenance task when modified by 
the judgement of experienced maintenance technicians and project 
engineers. The logistics and operational concepts basing, maintenance 
organization and flying schedules are developed in detail for various 
scenarios and set down in explicit form for input to the simulation. 
The simulation attempts to generate the specified missions in the 
computer. Aircraft are turned and maintenance accomplished as they 
break or need service according to the task data specified in the input. 
Manning levels in the various work centers are constrained on successive 
runs until further reduction prevents^-the necessary number of missions 
from getting off. The pair of computer programs documented in this 
volume are used in the final stage of the maintenance manpower simula- 
tion modeling process. They integrate direct work center manning output 
from a number of simulation runs; add supervision, overhead requirements, 
and standard element manning; and apply appropriate grade spread criteria. 
The output^is a complete basic authorization document for maintenance 
manning, compatable with Air Force manpower authorization procedures. 
The program may be exercised to obtain manning for any proposed flying 
rate within the simulated range, and has an option for adjusting existing 
equations with new data representing updated or alternative conditions. 
The methodology and models have been successfully applied on the A-10 
program and are being considered for implementation on other aeronautical 
systems in or entering development. Work is underway in conjunction with 
Air Force Logistics Command to incorporate this methodology into a 
system for total logistics tradeoffs and life cycle costing. 
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34. Mullen, P. A., & Joyce, R.P. Demonstration of Fully Proceduralized 
Job Performance Aids and Matching Training, AFHRL-TR-74-69 , August 
1974. AD-A002 147. 

Sixteen airmen with no background in electronics were given four 
weeks of job-oriented training and then observed as they performed 
maintenance while using fully proceduralized job performance aids 
(FPJPAs). Half of the airmen had the high aptitude usually required 
for training in electronics. The other half had significantly lower 
aptitudes. There is some evidence that most of the airmen were below 
average in reading ability. This effort had two purposes. One was 
to provide interested military and civilian personnel a preview of 
the maintenance performance of minimally ^irained airmen using FPJPAs, 
Another was to shake-down or debug the FPJPAs and the matching train- 
ing program prior to a major experimental comparison of alternative 
types of technical data and training for maintenance. The airmen were 
observed by the experimenter and a continually changing group of 
interested observers as they performed 14 maintenance tasks on a 
Doppler Radar set (AN/APN-147) and its Computer (AN/ASN-35) , Ten of 
the problems involved troubleshooting, two involved removal and 
replacement of hardware and two involved soldering. With an average 
of 1.5 ^'assists" per troubleshooting problem, the airmen were able to 
solve the problems in average times (for various problems) ranging 
from 18 to 59 minutes. The 90 "assists" were needed because of 
reading errors (52), misuse of multimeter (20), lack of manual 
dexterity (6), misuse of oscilloscope (5), and miscellaneous errors 
(7).^ On the troubleshooting tasks, the high aptitude trainees scored 
consistently better than the medium aptitude trainees in both time 
and number of 'assists; however neither of these differences were 
statistically significant. The high aptitude group was able to 
complete 10 of their 30 problems without an assist while the medium 
aptitude group completed only two problems without an assist. On the 
remove and replace tasks, both groups proceeded smoothly and, with one 
exception, without error. In the soldering tasks, the medium aptitude 
group was slower but their products were of higher quality than the 
high aptitude group. Although the airmen did not exhibit the errorless 
performance that had been desired, their performance should be considered 
as at least promising. Causes for .many of their deficiencies were 
identified and can be corrected. In general, the presence of the high- 
ranking observers seemed to disturb and slow the performance of the 
airmen. Also, it should be noted at no baseline is available to 
^ compare the performance of these airmen with the performance of con- 
ventionally trained airmen using conventional technical data, A 
training program about two weeks longer than the four week course used 
in this demonstration will be necessary to achieve satisfactory per- 
formance of maintenance tasks, including troubleshooting, while using 
FPJPAs, The FPJPAs must be completely verified to insure clarity and 
accuracy. Although personnel with high aptitude for electronic main- 
tenance probably will out perform those with lower aptitude, satisfactory 
eithe^hl^^ performance probabl, can be obtained from personnel of 
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35, Pieper, W.J., Pinkus, A.L. , & Thomas, D.L. Computer Generated Trouble- 
shooting Trees; Application and Tryout. AFHRL-TR-74-20 (l) , November 

. 1974. AD-A004 634. 

This report describes the results of a project to refine and test a 
procedure for developing troubleshooting trees by computer. In an 
earlier project conducted by NASA, a computer program and data pre- 
paration procedures were developed. However, the program and 
procedures were not fully tested. In this project, the procedures 
and program were refined, adapted, and tested. They were tested by 
using them to develop troubleshooting trees for a moderately complex 
electronic system. The troubleshooting trees were then evaluated by 
inserting faults into the equipment and using the trees to isolate the 
faults. Although the trees did not lead to isolation of 100% of the 
faults, the results did indicate that development of troubleshooting 
trees by computer is feasible. However, further refinements of the 
process are required before it can be used operationally. Additional 
information on the development and use of the computer program is 
given in AFHRL-TR-74-20 (II) . 

36, Pieper, W. J . , & Pinkus, A.L. Computer Generated Troubleshooting Trees ; 
The Program. AFHRL-TR-74-20 (II) , July 1974. AD-785 139. 

This volume describes the development, use, and tryout of a computer 
program to prepare troubleshooting trees by computer. The program 
inputs information on the system data flow, component reliabilities, 
and costs of available tests. An iterative process is then used to 
select the most efficient sequence of tests to isolate all possible 
faults. This is accomplished by computing an index of information 
gsClned per unit cost (IGUC) for each test. The test with the highest 
IGUC is selected as the first test in the tree. The IGUCs are then 
recomputed for the remaining tests and the test with the highest IGUC 
is added as the next step in the tree. The process is continued until 
a tree is developed which will isolate all faults in the system. The 
procedures and program were tested by using them to develop trouble- 
shooting trees for a 300 component electronic system. The trouble- 
shooting trees developed were tested by using them to isolate faults 
inserted i^L.^ Che equipment. Although the trees did not lead to 
isolation of 100% of the faults, the results did indicate that 
computer development of troubleshooting trees by computer is feasible. 
However, further refinements of the process are required before it can 
be used operationally. 

37, Potter, N.R. , & Dieterly, D.L. Methods for Predicting and Assessing; 
the Impact of Technology on Human Resources Parameters; Report of the 
Literature. AFHRL-TR-74-71, August 1974. AD-AOOO 051. 

A research objective of the Air Force Human Resources Laboratory is the 
development of methods for defining the components of innovative tech- 
nology and for measuring the effects of the incoming technology on Air 
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Force human resources • The human resource parameters of concern 
include manpower (e.g., numbers, job types, skill levels), training 
considerations, and cost data. Earlier investigations established 
the feasibility of using human resources data in design trade studies 
and of determining an graphically depicting the array of trade-off 
options available before inception of hardware design. The purpose 
of this review was to establish the current status of the methodology 
for forecasting and assessing technology and for quantizing human 
resource parameters with respect to the impact of incoming technologies. 
A search and critical analysis of the literature was undertaken to 
review the status of forecasting and assessing technology and of 
techniques for predicting the impact of technology on human resource 
parameters. The size of the literature base in the area of technological 
forecasting and technology assessment precluded the conduct of an 
exhaustive review. Rather, the determination was made that inclusion 
of selected references which described current methodologies in detail 
and which were judged as representative would better satisfy the 
objectives of the research effort. A total of 140 documents applicable 
to this effort were reviewed. Seventeen of these were identified as 
having major significance to the present effort. The review of the 
literature failed to provide solution to the problem of quantizing 
human resource parameters with respect to impact of incoming tech- 
nologies. However, the use of a normative forecasting technique was 
strongly supported by the literature. In particular, a relevance 
tree approach was the technique viewed as amenable to the problem of 
successive identification of increasingly finer components in an 
organized, structured manner. The relevance tree procedure known as 
a Design Option Decision Tree was. identified as practical for detailing 
a system to a level permitting the identification and assessment of 
human resource components for impact quantification. 

Scheffler, F.L., DaPolito, F.J., McAdams , R.L. , & Gee, M.J. Feasibility 
of Compu ter Processing of Technical Information on the Design of Instruc- 
tional Systems. AFHRL-TR-73-40, January 1974. AD-778 073. 

A study was made of the feasibility of a computer-based system to handle 
technical information pertinent to the design and use of instructional 
systems. The study took into account the information needs of both 
researchers and practitioners. Current and projected information needs 
of researchers and practitioners were determined from structured inter- 
views. Ten available computer-based information storage and retrieval 
systems which could serve as a basis for such a technical system were 
investigated. Question-answering computer systems based on artificial 
intelligence concepts were reviewed and considered in terms of providing 
a perscriptive interactive system. It was concluded that practicable 
generalized semantic information retrieval systems would require further 
developments. The design and implementation of a natural language 
automated information retrieval system encompassing rudimentary features 
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of a prescriptive system is feasible, both in terms of the technical 
resource provided and its cost effectiveness. An on-line interactive 
retrieval system was designed, and a model data base of approximately 
500 literature-derived comprehensive abstracts was implemented and 
tested to confirm the feasibility of the system. 

39. Schumacher, S.P., Swezey, R.W., Pearlstein, R.B., & Valverde, H.H. 
Guidelines for Abstracting Technical Literature on Instructional 
System Development. AFHRL-TR-74-13 , February 1974. AD-777 757. 

ri'idelines are presented for preparing abstracts of technical litera- 
L ce on instructional systems development (ISD) . Although specifically 
developed for abstracting information during the preparation of a 
technical data file on ISD, the guidelines are sufficiently general 
to apply to other areas in which abstracts of technical literature are 
desired. For abstracting purposes, the literature was divided into two 
categories: (1) Type I and (2) Type II documents. The first category 
includes opinion articles, methodological developments, evaluative 
^ summaries, literature reviews, and bibliographies. The second 

category includes statistical sampling studies, correlational research, 
and research studies in which variables are manipulated. Both types 
of abstract are prepared on the same general form. A sample of the - 
abstract form and examples of acompleted Type I and Type II abstracts 
are included. The sample form permits the most important characteristics 
of a document to be synthesized to facilitate the computerization of 
the technical data file. 

40. Schumacher, S.P. , Pearlstein, R.B., & Martinj P.W. A Comprehensive 
Key Work Index and Bibliography on Instructional System Development. 
AFHRL-TR-74-14, February 1974. AD-777 192. 

This report provides a bibliographic listing of the 2,692 items selected 
and abstracted for a basic file of technical information on instructional 
system development. Both reports of pertinent original research as well 
as summarizing and discursive articles are included. The items date as 
far back as 1953 and represent a subset of articles within the general 
topic selected for special relevance. In addition, a comprehensive 
index according to over 600 key words is provided to assist the user in 
finding items of immediate interesL 

41. Schumacher, S.P. , & Wiltman, S. A Compendium of Research and Develop- 
ment Needs on Instructional System Development. AFHRL-TR-74-15 , 
February 1974. AD-777 196. 

This report provides a comprehensive listing of specific needs for 
research on instructional systems development, as identified and noted 
by the authors of 2,692 related articles. The articles from which the 
needs were taken had been selected and abstracted for a basic file of 
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technical information on instructional system development. Because 
the items date as far back as 1953 some, of course, may no longer be 
relevant but no attempt was made to reflect degree of current relevance 
in this listing. The needs are categorized and listed by principal 
system development activities and, in turn, by more specific subsidiary 
concerns . 

Shriver, E.L. , & Foley, J. P. Evaluating Maintenance Performance; The 
Development and Tryout of Criterion R e ferenced Job Task Performance 
Tests for Electronic Maintenance. AFHRL-TR-74~57(II) , Part I, 
September 1974. AD-A004 845. 

The previous Volume, AFHRL~TR-74-57 (I) , recommended the development of 
criterion referenced Job Task Performance Tests (JTPT) for typical 
electronic maintenance activities. This Volume reports the develop- 
ment of a battery of such tests together with an appropriate scoring 
scheme for reporting the results of administering them. The develop- 
ment of a Test Administrators Handbook also is described. This battery 
is considered to be a model for future criterion JTPT development and 
is intended for both formal training and field use. The battery includes 
separate tests for the following classes of job activities: (1) equip- 
ment checkout, (2) alignment /calibration, (3) removal/replacement, (4) 
soldering, (5) use of general and special test equipment, and (6) 
troubleshooting. The Doppler Radar, the AN/APN-147 and Its computer 
the AN/ASN-35 were selected as a typical electronic system. This 
system was used as the test bed for this model battery. The soldering 
and general test equipment JPTP are applicable to all electronic tech- 
nicians. The other -tests of the battery apply to technicians concerned 
with this specific doppler radar system. Each class of activity for 
which JTPT were developed contains its individual mix of behaviors, but 
it is not mutually exclusive. There are dependencies among the classes. 
As a result a four level hierachy of dependencies can be stated: (1) 
checkout, removal/replacement , and soldering; (2) use of general and 
special test equipment; (3) alignment /calibration; and (4) trouble- 
shooting. For example, troubleshooting may include all the activities 
mentioned before it. Due to the diverse character of the various 
mixes of behaviors involved in each class of maintenance activity, a 
single score report of test results would be meaningless. A profile of 
test results therefore was developed which provides for an individual 
cell for each test problem. The tests are structured so that each 
problem produces a product. The results for each problem is reported 
in terms of a go, no-go score. Either the test subject produces a 
satisfactory product or he does not. Where time is important, he must 
produce the satisfactory product in a specif ied time. Although process 
may be valuable as a diagnostic tool, it is not considered as an 
appropriate factor for scoring purposes. The hierachy of dependencies 
mentioned previously has implication for the order in which tests are 
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administered as well as for diagnostics. For example, since trouble- 
shooting includes the use of test equipment and other activities in 
the hierachy, logic would dictate that administration of the tests 
for the sub-activities would precede the troubleshooting tests and 
that a test subject would not be permitted to take the troubleshooting 
tests until he had passed these other subtests • Due to the unavail- 
ability of a sufficient number of experienced test subjects at the 
time of the tryout of the JTPT battery, the tryout was not as extensive 
as planned. The limited tryout did indicate that the tests as developed 
are administratively feasible. There continued use, no doubt, would 
result in further modifications and polish. The report also includes a 
discussion of several implementation considerations and suggestions. 

43. Shriver, E.L., Hayes, J.F., & Hufhand, W.R. Evaluating Maintenance 
Performance: Test Administrator's Manual and Test Subject^s 
Instructions for Criterion Referenced Job Task Performance Tests 
for Electronic Maintenance. AFHRL-TR-74-57(II) , Part II, January 
1975. AD-A005 785. 

The purpose of this document is to furnish a complete copy of the 
Test Subject's Instructions and the Test Administrator's Handbook 
for a battery of criterion referenced Job Task Performance Tests 
(JTPT) for electronic maintenance. Part I of Volume II of this series 
of documents, AFHRL-TR-74-57 , reports and describes the development 
and tryout of this battery of tests. 

44. Shriver, E.L., & Foley, J. P. Evaluating Maintenance Performance: The 
Development of Graphic Symbolic Substitutes for Criterion Referenced 
Job Task Performance Tests for Electronic Maintenance. AFHRL-TR- 
74-57(111), November 1974. AD-A005 296. 

An in-depth review of the literature reported in AFHRL-TR-74-57 (I) , of 
this series of documents strongly reiterated the fact that paper and 
pencil tests of job knowledge and electronic theory tests have very 
poor criterion-related or empirical validity with respect to the ability 
of electronic maintenance men for performing their job tasks. As a 
result, a battery of criterion referenced Job Task Performance Tests 
(JTPT) was developed and tried out and results were reported in AFHRL- 
TR~74-57(II) . The battery included tests for the various job activities 
performed by electronic maintenance technicians such as checkout, 
align/adjust, remove /rep lace , soldering, use of general and special test 
equipment, and troubleshooting. Past experience with tests similar to 
these criterion referenced JTPT has indicated th^t even though criterion 
referenced JTPT were recognized as being superior by many training 
people, paper and pencil tests were substituted because they were more 
easily and cheaply developed and administered. The JTPT developed 
required the test subject to use actual equipment and they could only 
be administered to very small groups; in some cases, to only one subject • 
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Graphic symbolic substitutes would probably overcome these admin- 
istrative problems. But such substitutes must have high empirical 
validity. Most previous attempts to develop such tests resulted in 
low validity. This volume describes another attempt to develop a 
battery of graphic symbolic substitutes of improved validity A 
review of previous attempts resulted in a hypothesis that previous 
attempts had weaknesses in realism that could possibly be rectified 
The successful accomplishment of most maintenance tasks must follow* 
a main line procedure or strategy. But this main line usually is 

cluttered with distraction and subprocedures which interfere with 
the accomplishment of the main line procedure or strategy. For 
example, when troubleshooting, the technician must usually interrupt 
his strategy several times to set up his test equipment and to obtain 
check point information. Unless he is well organized and very 
persistent in following his strategy, he may los.e track of his strategy. 
Even If he stays on the main line, he may gather faulty information 
from his test equipment which will prevent him from finding the trouble. 
Based on this rationale, it was concluded that previously developed 
symbolic tests such as the tab test did not provide such clutter It 
was hypothesized that symbolic substitute tests could be developed that 
would retain a large amount of realistic task "clutter" and that such 
tests would have higher empirical validity than previously developed 
symbolic tests. In this effort, a battery of symbolic tests was 
developed including a companion symbolic test for each of the job 
activities for which a criterion referenced JTPT had previously been 
developed. Based on two limited validations, all of the graphic 
symbolic tests, with the exception of the symbolic test for soldering, 
indicated sufficient promise to justify further consideration and 
refinement. All of these promising symbolic tests should be given 
more extensive validations using larger numbers of experienced subjects, 
ihe validation of any such symbolic tests requires the administration 
of a companion JTPT as a validation criterion. As a result, a valida- 
tion IS an expensive process in terms of equipment and experienced 
raanpower. The troubleshooting symbolic tests require the most extensive 
refinement. Several suggestions are made for improving their empirical 
validity. 

Shriver, E.,L. , Hayes, J.F., & Hufhand, W.R. Evaluating Maintenance 

_Performance: A Video Approach to Symbolic Testing of Electr onics 

Maintenance Tasks. AFHRL-TR-74-57rTVV^ .Tnly iQ7A Ar-A005 297. " 

This volume reports an effort to use the video med '.a as an approach for 
the preparation of a battery of symbolic tests thaf would be empirically 
valid substitutes for criterion referenced Job Task Performance Tests. 
The development and tryout of such criterion refers, ced tests and 
promising graphic symbolic substitute tests are repf^rted in previous 
volumes. The graphic symbolic tests require the str?age of a large 
amount of pictorial information which must be searched rapidly for dis- 
play. At the time this video effort was started, no completely 
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satisfactory way had been found for rapidly searching and displaying 
such information. In addition, some dynamic displays would have been 
desirable for those graphic symbolics where as all of the original 
graphic pictorials are static. The anticipated results that the video 
media would provide satisfactory and economical solutions to these 
problems, did not materialize. From a testing point of view symbolic 
tests place the test subject in a passive, evaluative role of watching 
someone else perform each task. Success in this passive role does not 
insure his success in the active role of performing the same task. 
Also, the development costs of the video tests proved to be very high in 
terms of video equipment and test development time. To obtain video 
materials of acceptable quality, would require both quality video equip- 
ment and studio production facilities. As a result, it was recom- 
mended that video should not be further considered as a testing medium 
for performance analogues and that future efforts should be aimed at 
improving and refining graphic s.ymbolic substitute tests. 

Tetmeyer, D.C. Estimating and Controlling Manpower Requirements for 
New Systems: A Concept and Approach. AFHRL-TR-74-31 , April 1974. 
AD-778~838. 

The Air Force faces qualitative, quantitative and budgetary limitations 
on the number of personnel by AFSC that- can be provided to maintain new 
weapon systems. A systems concept is proposed for manpower prediction 
and control during the development cycle, using automated data systems 
and simulation techniques. A plan is presented for a feasibility 
demonstration during validation phase on the A-X Tactical Aircraft. 

Tetmeyer, D.C, & Moody, W.D. Simulating Maintenance Manning for New 
Weapon Systems: Building and Operating a Simulation Model. AFHRL-TR- 
74-97(11), December 1974. 

The Air Force Human Resources Laboratory, in conjunction with Headquarters 
Air Force Systems Command and the Aeronautical Systems Division, has 
developed and implemented a new method for determining and evaluating 
the maintenance manning required for new weapon systems. Key features 
of the new approach are: 1. Manning requirements are determined by 
dynamic simulation of maintenance work to support specific operational 
scenarios. The number of people needed to support the same aircraft 
and flying hour program varies greatly with basing and deployment, 
mission types, and sortie length. 2. Manning is based on required 
availability, not just on the sum of manhours worked. While worktime 
is a major consideration, equipment does not break on schedule and 
combat aircraft cannot always fly on a schedule that provides an even 
workload. 3. Maintenance frequencies and job requirements for a new 
system are extrapolated from Air Force experience with similar equip- 
ment, adjusted for differences in design and environment. The total 
time a man is tied up on a job is explicitly modeled, not just "wrench 
turning" time. 4. Output is in a format compatible with the Air Force 
manpower authorization system. This approach*- provides more realistic 
and useable estimates of manning requirements and focuses on how both 
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a contractor's design and Air Force procedures contribute to mainte- 
nance cost. As soon as the new aircraft configuration can be defined, 
comparable equipment and design differences are identified at suD- 
system level. Maintenance data on comparable equipment are processed 
through a series of specially designed programs. These data are the 
basis for developing initial estimates of the frequency, time, and 
resources for each maintenance task when modified by the judgement of 
experienced maintenance technicians and project engineers. The 
logistics and operational concepts, basing, maintenance organization 
and flying schedules are developed in detail for various scenarios 
and set down in explicit form for input to the simulation. The 
simulation attempts to generate the specified missions in the computer. 
Aircraft are turned and maintenance accomplished as they break or need 
service according to the task data specified in the input. Manning 
levels in the various work centers are constrained on successive runs 
until further reduction prevents the necessary number of missions 
from getting off. The results of various simulation runs are integrated 
in a final series of computer programs that computes manning in each 
work center for any flying program within the simulated range, adds 
authorized overhead, adjusts for grade level limitations, and produces 
a complete basic authorization document for maintenance manning in the 
required format. The methodology and models have b en successfully 
applied on the A-10 program and are being considered for implementation 
on other aeronautical systems in or entering development. Work is 
underway in conjunction with Air Force Logistics Command to incorporate 
this methodology into a system for total logistics tradeoffs and life 
cycle costing. This volume describes the procedures to build a main- 
tenance simulation data base for a new aircraft, to run it with the 
Logistics Composite Model (LCOM) computer program, and to use the results 
to predict the maintenance manning required for a new aircraft flying a 
specific operations scenario. Computer programming knowledge is not 
necessary to follow and apply this instruction. The procedures 
presented have been tested and used at the Aeronautical Systems Division 
of Air Force Systems Command ±n building and operating simulation models 
of A~10 and A7 aircraft. 

Whalen, G.V. , & Askren, W.B. Impact of Design Trade Studies on System 
Human Resources. AFHRL-TR-74-89 , December 1974. AD~A009 639. 

This study was undertaken to accomplish two objectives. The first 
objective was to identify and classify the characteristics of con- 
ceptual design trade studies that have high potential impact on human 
resource requirements of Air Force weapon systems. The approach used 
was a case history review and analysis of 129 F~15 aircraft design 
trade studies. The analysis indicated that the avionics system 
demonstrated the greatest potential Impact on human resources. It was 
also found that trade studies dealing with design alternatives that 
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encompass widely different technologies have substantial impact on 
human resources. The types of human resources data (HRD) most 
influenced by alternative design options were maintenance task times 
and personnel costs. The second study objective was to determine the 
accuracy of using subjective estimates as a technique for deriving 
the HRD impact of trade study options. Using only engineering infor- 
mation for six avionics subsystems, from the conceptual design phase. 
Air Force maintenance technicians made subjective estimates of the 
impact of the designs on selected HRD items. It was found that 
technicians can make highly accurate estimates of the amount of time, 
the Air Force occupational specialty, the level of technical skill 
and the number of personnel needed to perform field maintenance tasks. 
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